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1. New Credentials - New 
Learning Opportunities 


Human beings are compulsive learners. 
Throughout our ever-longer lives we are 
constantly processing knowledge. New facts, 
new experiences and new insights please our 
nerve cells. While we sleep at night our brain 
is archiving what we have learned that day. 
These neurological facts make us all lifelong 
learners, independent of formal enrolment. 


“Shorter-term educational 
experiences are on the 
rise, often using online 
teaching, learning 

and assessment and 
issuing digitally signed 
credentials.” 


Strong demographic and economic factors 
coincide with this natural desire to learn. The 
United Nations Educational, Scientific and 
Cultural Organization (UNESCO) estimates 
that, by the year 2030, nearly 200 million 
young people would like to enter higher 
education, mainly in the fast-growing global 
south. Current and future workforces will 
require frequent training and retraining to 
adapt to changing labor markets.' 
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Meanwhile, discussions on higher education 
tend to focus on long programs of formal, 
initial education, provided by traditional 
colleges and universities. These long 
programs are subject to state regulation, 
inspection, quality assurance and 
accreditation. Less attention is given to the 
wealth of learning taking place through 


shorter-term educational experiences (e.g., 
mini-, micro-, nano-credentials) offered inside 
and outside formal education. 


Shorter-term educational experiences 

are on the rise. They often use online 
teaching, learning and assessment. Many 
providers issue digitally signed credentials, a 
noticeable phenomenon in traditional higher 
education as well. These novelties raise all 
sorts of questions on quality, authenticity, 
transferability, sustainability and potential 
fraud. 


This current paper focuses on quality aspects 
of shorter-term educational experiences. 
What features determine quality? What type 
of quality reviews would be most suitable? 
Are accreditation and quality assurance 

as we know them still relevant? What 

role for the Council for Higher Education 
International Quality Group (CHEA/CIQG) 
Quality Platform, one approach to addressing 
the challenge of assuring quality for these 
experiences? 


The preliminary responses in this paper 
draw on the discussion of a Working 

Group convened by CHEA/CIQG and the 
American Association of Collegiate Registrars 
and Admissions Officers (AACRAO) in 
Washington DC on 13 March 2019.2 The 
Working Group focused on the current state 
of shorter-term educational experiences and 
efforts to assure quality in this emerging 
sector: Who are the providers? Are numbers 
growing? What do we know about assuring 
quality? 


2. A Wide Variety of 
Providers 


Shorter-term educational experiences vary 

in content, length and mode of delivery; they 
vary most significantly in type of provision. 

In recent years, there is a clear shift towards 
these alternative providers, notably the online 
platforms that function as intermediary 
between content providers and learners. 
These platforms are very much in the public 


1 Nearly half of the today’s jobs are going to either disappear or change significantly through automation and the use 
of artificial intelligence. Source: OECD Employment Outlook 20179:The Future of Work. 


2 The Working Group consisted of nine colleagues from four countries and was chaired by Peter van der Hijden, 


Higher Education Expert. 
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eye, whereas other longstanding providers of 
shorter-term educational experiences, such 
as professional associations, are not so widely 
discussed. Six big categories of providers can 
be distinguished: 


Higher Education Institutions 


Most higher education institutions are not 
currently engaged with the demand for 
shorter-term educational experiences, but 

a good number of them are discovering 

the growing segment of ad hoc learners 
who appreciate the academic context 

and prestige. The micro-credentials these 
institutions offer may be stand-alone or part 
of stackable degree programs. They may be 
delivered in partnership with other types of 
providers listed below. They may or may not 
be subject to direct or indirect quality reviews 
and accreditation. By one estimate, for 
example, 906 universities around the world 
are offering massive open online courses 
(MOOCs).3 


Online Platforms 


Online Platforms offer easy access to 
education, mainly through massive open 
online courses (MOOCs), which are mostly 
produced by traditional higher education 
institutions. The most prominent platforms 
are Coursera, Udacity and edX. In addition 


“Shorter-term educational 
experiences vary in 
content, length and mode 
of delivery; they vary 
most significantly in type 
of provision.” 


MAGN) 


to these U.S. platforms at the forefront of 
the MOOCs movement, the European MOOC 
Consortium (EMC)4 has recently launched a 


3 httpos://www.classcentral.com/universities. 


Common Microcredential Frameworks. Just 
five MOOCs - Coursera, edX, XuetangX, 
Udacity and FutureLearn - enrolled more 
than 68 million people in 2018*% These 
platforms and the offerings are often free or 
change a fee for certification of work that has 
been completed. MOOCs are often among 
the first to engage traditional institutions 
through various partnerships such as those 
mentioned above. 


Companies 


Most companies of a certain size provide 
some kind of in-house training. These are 
typically big corporations that develop full 
degree programs or universities or training 
institutes of their own. Corporate universities 
typically provide job- or company-specific 
training mainly for the managerial personnel 
of the parent corporation. Examples include 
Charles Schwab University, Disney University, 
General Electric’s Crotonville Campus, 
McDonald’s Hamburger University, Motorola 
University, Oracle University and University 
of Toyota, Microsoft certifications, CISCO 
certifications and Mozilla Open Badges, to 
name just the most prominent ones. 


Professional Associations 


Professional associations serve their members 
and society by organizing continuous 
professional development courses and 
associated licensing. This has been going 

on for many years. The American Medical 
Association (AMA), for example, has 
developed a nationwide accreditation system 
for Continuing Medical Education providers, 
the AMA Physician’s Recognition Award 
(PRA). The World Federation of Medical 
Education (WFME) provides certification and 
accreditation of the suitability of medical 
education programs and the competence 

of medical schools in the delivery of 

medical education. The American Society 

of Mechanical Engineers (ASME) provides 
assessment and certification. There is a 


4 Consisting of FutureLearn, France Université Numérique, OpenupEd, Miriadix and EduOpen. 


5 https://emc.eadtu.eu/images/Press_release_ European MOOC_Consortium_launches_a_Common_Microcredential_ 


Framework.pdf. 


6 httos://www.classcentral.com/report/mooc-stats-2018/. 
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host of certifications for accountants (e.g., 
in the United States, the Certified Public 
Accountant - CPA) and this is the case for 
most professional disciplines. 


Associations of Leisure 


Leisure organizations, such as sports clubs, 
provide all kinds of training to their members 
resulting in various types of certificates. The 
Boy Scouts of America (which now admits 
girls) and the Girl Scouts of America, for 
example, award hundreds of badges for 
learning achieved in sports, crafts, science, 
trades, business and future careers. 


Social Media 


Social media such as Google, Facebook, 
YouTube and Wikipedia transmit learning 
content and increasingly do so in education 
and training-like formats, including self- 
assessment arrangements. LinkedIn Learning, 
for example, offers MOOCs taught by 
industry experts in software, creative and 
business skills. 


3. Expected Growth 


The offer and take up of the various kinds 

of shorter-term experiences are expected to 
grow exponentially in the years ahead. Formal 
education will continue to play a major role 

in the learning landscape as initial provider 
and overall standard setter. Many institutions 


“The offer and take up 

of the various kinds of 
shorter-term experiences 
are expected to grow 
exponentially in the years 
ahead.” 


of higher education will, however, not be able 
or willing to engage fully with shorter-term 
educational experiences, thus allowing other 
parties to step in or extend their role, not 
least because of the technological advances 
made in recent years. The fast growth will be 
both demand-driven and technology-driven. 
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Demana-driven growth will come from 
citizens worldwide who have no other choice 
than to regularly refresh their capacities. 
Individuals nowadays need a steady flow 

of new knowledge they can learn to handle 
with the right skills and wider competences, 
enabling them to cope successfully with the 
uncertainties of life and the labor market. 
They seek the latest technical knowledge 

in a given field, but also soft skills to better 
interact with others, capacity for critical and 
systemic thinking and digital skills to process 
data flows, use artificial intelligence and 
collaborate with robots. 


Technology-driven growth will result from 
the maturing of online learning provision 
and the emerging standardization of digital 
certification techniques. Online learning 

and blended learning are ubiquitous to 

the extent that the distinction between 
traditional education and distance education 
is starting to blur. Campus universities 

offer all kinds of online services, from 
admission to examination, whereas online 
universities proudly promote physical contact 
opportunities between students (peer to 
peer) and staff (guidance, seminars, summer 
courses). 


4. Digitization of 
Credentials 


Growing numbers of education providers 
issue digitally signed and certified versions 

of their credentials, in addition to or 

instead of the paper formats. The use of 
electronic formats raises questions but also 
provides solutions for effective student data 
management, including but not exclusively 
through the use of block-chain techniques. 
eLearning Industry describes “digital 
credentials” as a digital form of a physical 
credential across a diverse array of areas, 

not only higher education but also business 
and government (lafrate, Mark, November 6, 
2017’). The Credential Engine, a U.S. nonprofit 
organization that centralizes credentials at 
multiple levels, lists 719 digital credentials 
entries from higher education, businesses and 
other organizations and quality assurance 
bodies.® 


7 https://elearningindustry.com/guide-to-digital-badges-how-used. 


8 https://credentialfinder.org/. 
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The growing impact of digitalization on 
credentialing worldwide is discussed 
notably within the Groningen Declaration 
Network. Signatories of the 20/2 Groningen 
Declaration include National Student Data 
Depositories of countries such as the United 
States, China, France, the Netherlands, 
South Africa, Australia and India as well as 
growing numbers of university networks and 
certification startups. As described on its 
website: “The Groningen Declaration Network 
seeks common ground in best serving the 
academic and professional mobility needs 
of citizens world wide by bringing together 
key stakeholders in the Digital Student Data 
Ecosystem at its Annual Meeting. We make 
digital student data portability happen. 


“Growing numbers of 
education providers 
issue digitally signed and 


certified versions of their 
credentials.” 
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Citizens worldwide should be able to consult 
and share their authentic educational data 
with whomever they want, whenever they 
want, wherever they are.”® 


Of particular interest in the U.S. context 

are the Comprehensive Learner Records 
(CLRs), which aim at the development and 
implementation of a single learner record 
across a broad number of American colleges 
and universities. The CLRs seek to capture, 
record and communicate learning when 

and where it happens in a student’s higher 
education experience. This includes learning 
outcomes from courses, programs and 
degrees, as well as experience outside the 


9 httos://www.groningendeclaration.org/. 


classroom that help develop career-ready 
skills and abilities. 


Of particular interest in the European context 
is the New Europass Framework, which 
envisages the creation of learner owned, third 
party certified, digitally signed credentials, 

to be stored on personal computers or more 
conveniently in public or private run “wallets” 
accessible with learners’ permission only." 


5. Determining Quality 
- Basic Features for All 
Types of Provision 


The expected growth of shorter-term 
educational experiences inevitably raises 
questions in terms of intrinsic quality 

and quality review: quality assurance and 
accreditation. Will the longstanding norms 
and expectations of quality be effective for 
shorter-term learning experiences? What, 
specifically, needs to be examined to judge 
quality? Might the development of new forms 
of quality review specifically tailored for 
shorter-term experiences be desirable and 
even anticipated? 


“Intrinsic quality” of education is hard to 
define and even harder to measure. There are, 
however, a number of features that indicate 

a certain quality (as proxies providing 
circumstantial evidence). If the following 
basic features” are in place, in whole or in 
part, a great deal is said about the quality of 
what is on offer: 


Typology 
e Mission statement of the provider 


* Level referenced against a 
qualifications framework® 


10 Developed by AACRAO and NASPA: Association of Student Affairs with the support of the Lumina Foundation. 


11 Decision of the European Parliament and of the Council of 18 April 2018. 


12 The features listed here derive from: EADTU, Quality Assessment for E-Learning, a Benchmarking Approach, 
Heerlen, 2012; Academic Partnerships, A Guide to Quality in Post-Traditional Online Higher Education, Dallas 2014; 
EHEA Secretariat, Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG 


2015) Brussels, 2015; JRC Science for Policy Report, Va/idation of Non-formal MOOC-based learning’, Brussels, 2016; 
ENIC-NARIC, Oops a MOOC! Dealing with eclectic learning in credential evaluation, February 2018; Hochschulforum 

Digitalisierung Position Paper, Bologna Digital, Berlin, May 2018; CHEA-CIQG, New Approaches to Judging Quality in 
Higher Education, Washington, 2018 


13 Sector-, national, international, e.g., the Tuning Degree Profiles and the European Qualifications Framework in 
Europe, the Lumina Degree Qualifications Profiles in the United States and the upcoming World Reference Levels 
CWRL) of UNESCO. 
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¢ Profile indication (e.g., research 
oriented, profession oriented, general 
interest oriented) 


¢ Workload indication (average time or 
credits) 


¢ Learning outcomes descriptors 
(knowledge, skills, degree of 
responsibility and autonomy) 


¢« Summative assessment 


* Certificates, diploma supplements, 
badges etc. acknowledging learning 


Appreciation 
¢ Uptake among learners 


« Higher education institutions accept 
the credential as part of accredited 
degree programs” 


¢ Employers or employers’ associations 
recommend the credential for hiring 
and promotion 


¢ Professional associations accept the 
credential for licensing purposes or 
continuing professional development 


Reputation 


« Past performance of the provider in 
education and research (e.g., rankings 
and citations) 


« Partnerships and collaborations of the 
provider (e.g., leagues) 


Over-emphasizing appreciation and 
reputation works against newcomers on 
the education market, but this type of 
information is an inevitable part of the 
equation. 


Learner information 


« Identification 


e Partnerships and collaborations of the 
provider (e.g., leagues) 


It should be stressed that the basic features 
of quality listed above apply to all modes 

of delivery of educational experiences at all 
levels, be they of longer or shorter duration. 
The performance standards against which 
the features are measured will vary inevitably 
according to the norms defined by the 
institution, the sponsor and the reviewer. 


For shorter-term educational experiences 
the same basic features would therefore 
apply, but one would expect more effort 
going into s/tuating the experience in the 
overall education landscape. The exact level 
of the experience, possible prerequisites and 
the veracity of the credential delivered will 
also raise more concern and require more 
scrutiny, at least in the beginning, compared 
to traditional degree programs. What matters 
most in all offerings is the learning outcomes, 
expected from and acquired by the students. 


“Will the longstanding 
norms and expectations 
of quality be effective 
for shorter-term learning 
experiences?” 


BVWOONss 


If the shorter-term educational experience 

is online or blended, that also may require 
specific knowledge and expertise on the part 
of the provider and the accreditor e.g., as 
regards online didactics. Mechanisms need to 
be in place as well to ensure that the learner 
information is correct and related to that 
individual only. 


The features of quality as proposed in this 
section do not measure quality directly, but 
they are very informative proxies. A learner, 
an employer, a sponsor or an admission 
officer surely will want to know, for example, 
what learning outcomes are envisaged at 
what level and how other employers and 
other institutions appreciate the credential 
holders so far. 


14 Purposely designed as such or through the recognition of prior learning. 


COUNCIL FOR HIGHER EDUCATION ACCREDITATION/CHEA INTERNATIONAL QUALITY GROUP | 10 


6. Nine Review Types 


Quality reviews can take very different 
formats, from purely formative and internal 
to summative and external. Publication 
strategies will differ accordingly. Nine review 
types are identified below. Some review 
types may be better suited to deal with the 
high number and variety of shorter-term 
educational experiences. 


Self-assessment 


Providers carry out periodical self- 
assessments and use the outcomes for 
internal purposes or as input for external 
evaluations. 


Peer Review 


Providers invite peers to review their 
performances. 


Benchmarking 


Providers use the outcomes of self- 
assessments and peer reviews to benchmark 
their performance against each other. 


External Evaluation and Audits 


Providers call in external evaluators or 

may be obliged to do so by their public or 
private sponsors. If shorter-term educational 
experiences (mini-, micro-, nano-credentials) 
are provided by higher education institutions 
one would expect these types of offering to 
be covered by the same periodical external 
evaluations at program or institutional level. 
Evaluation and audit results may be published 
in whole or in part.'® Accreditors themselves 
may also be subject to reviews, publishable in 
whole or in part." 


Provider Appreciation 


Providers may publish and update lists of 
third-party micro-credentials, which they 
accept as part of their degree programs and 
specify the conditions under which credits 
could be awarded.” 


Employer Appreciation 


Individual employers or employer 
associations may publish and update lists of 
micro-credentials they recommend for hiring 
or promotion purposes. 


Professional Appreciation 


Professional associations of lawyers, medical 
doctors, nurses, engineers etc. could publish 
and update lists of micro credentials they 
accept for licensing purposes or recommend 
for continuous professional development. 


Crowd Assessment 


Leaners can share their appreciations of 
micro-credentials on social media or on 
dedicated platforms of trusted peers. 


Comparative Assessment of 
Learning Outcomes 


Learning outcomes achieved by graduates 
in given subject areas could be tested and 
compared across institutions and countries 
and results thereof could be published 

in a Program for International Student 
Assessment (PISA) study fashion. 


It is unlikely that the current system of 
external evaluations and audits by teams 

of peers can be maintained for all sorts of 
short-term provisions. A mixed use of review 
types is more likely, whereby employers, 
professional associations and crowd 
assessment would probably take the lion’s 
share in terms numbers and frequency of 
reviews published. 


15 In Europe entire reports are published on the Database of External Quality Assurance Results - DEQAR. 


16 In the United States by CHEA and the US Department of Education, in Europe by EQAR. 


17 Cf. the United States, AACRAO’s Transfer Credit Practices (TCP) and ACE Credit Accepting Institutions, the 2016 
Regulation for the recognition of MOOCs in India and Recommendation 4 of the March 2018 Position Paper Bologna 


Digital. 


18 See the reports of the Assessment of Higher Education Learning Outcomes (AHELO) project of OECD, completed 
in 2012 and the university-led Comparing Achievements of Learning Outcomes in Higher Education in Europe 
(CALOHEE) pilot supported by the European Commission, entering a new phase in 2020. 
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Virtuous Circle of Trust 


Public information resulting from any given 
combination of these eight types of review 
may incite various categories of stakeholders 
to trust each other’s judgments and thus 
create a virtuous circle of trust among the 
wider community of users and providers. 


7. Quality Review in 
Transition 


The expected increases in the volume, 
diversity and uptake of shorter-term 
educational experiences will have 
consequences for the organization of quality 
review inside and outside formal higher 
education. 


Accreditation and quality review of higher 
education institutions in the future should 
encompass all learning provision at all 
levels, from foundation courses, to bachelor, 
master, doctorate and continuing education. 
Naturally this would include all shorter-term 
educational experiences delivered by these 
institutions. 


“Over-emphasizing 
reputation works against 
newcomers on the 
education market, but 
this type of information Is 
an inevitable part of the 
equation.” 
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Expanding the scope of accreditation to 
shorter-term educational experiences would 
continue the historic mission of accreditation 
and quality assurance of upholding standards 
in higher education in respect to the quality 
features outlined in this note. Accreditors and 
quality assurance bodies and their networks 
would continue to identify those standards, 
debate and promote them in dialogue with 
stakeholders inside and outside higher 
education. 


Alternative providers of shorter-term 
educational experiences may also learn to 
see the benefits of addressing the quality 


features for their offerings and having them 
reviewed (periodically in series) by trusted 
accreditors and quality assurance bodies. 
Campaigns could be organized to raise 
“Quality reviews can take 
very different formats, 
from purely formative and 
internal to summative and 


external.” 
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the quality awareness among those non- 
traditional providers. Learners may also 
decide to vote with their feet (or mouse) 
when they experience or read that certain 
offering is sub-standard. 


8. The CHEA/CIQG 
Quality Platform 


The CHEA/CIQG Quality Platform is one 
example of how an alternative form of 
quality review, deriving from but grounded 
in more traditional accreditation and quality 
assurance, could play a pivotal ro/e in 
promoting quality review of shorter-term 
educational experiences. The CHEA Quality 
Platform, developed by CHEA/CIQG in 2014 
and adapted to digital credentials in 2018, 
can bring together providers of the shorter- 
term educational experiences to test and 
explore what is effective in assuring quality in 
the digital credential sector. 


The Platform is an outcomes-based review 
of digital credentials such as digital badges, 
digital certificates, nano-degrees and 

other micro-credentials. These credentials 
are offered by traditional and alternative 
higher education providers for shorter 

term educational experiences. The review 

is conducted by a panel of experts who 
examine digital credentialing to assure that 
the credentials (1) include clear expectations 
of student learning at the post-secondary 
level, (2) provide reliable evidence that a 
student has demonstrated the skills needed 
to achieve the credential and (3) include 
evidence of acceptance by both the business 
sector and higher education institutions. 
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Other forms of quality review can be 
applied to digital credentials. These include, 
for example, organizations that review 

the design, development and workplace 
relevance of curriculum such as the U.S. 
Quality Matters or the American Council 
on Education. It could include European 
frameworks such as E-xcellence developed 
by the European Association of Distance 
Teaching Universities (EADTU) and 
OpenUpEd.” 


In most countries, government-based quality 
review remains focused on traditional higher 
education - colleges and universities. This 

is in part because these governments are 
assisting students and institutions with 
funding. Funding is less likely to be available 
for these newer types of educational 
experiences. However, as interest in relying 
on shorter-term educational experiences 
continues to go, governments may begin 

to focus on quality review in this emerging 
sector. 


9. Conclusions and Next 
Steps 


Several key points emerged from the Working 
Group discussion of digital credentials for 
shorter-term educational experiences and 
assuring quality: 


« The basic features of quality do not 
differ between long-term or shorter- 
term learning experiences. 


* New delivery modes, e.g., online and 
blended learning, will require some 
specialized knowledge and expertise 
to be developed among providers and 
accreditors. 


¢ Accreditors should include shorter- 
term educational experiences in their 
periodical review of institutions and 
programs. 


« Accreditors should organize 


19 https://eadtu.eu, https://www.openuped.eu. 


reviews for (series of) shorter-term 
educational experiences offered by 
alternative providers. 


Leaders in higher education, alternative 
providers, business and government are 
all involved in the digitization undertaking 
and, working together, need to continue to 
address: 


¢ What performance standards they 
wish to apply when reviewing the 
basic quality features of educational 
offerings, be they longer-term 
programs or shorter-term learning 
experiences? 


« What are the most efficient and 
effective mechanisms for quality 
assurance and accreditation in view 
of the expected exponential growth 
in shorter-term learning experiences? 
What is the role of traditional 
accreditation and quality assurance 
bodies next to reviewers from 
professional associations and crowd 
assessment platforms? 


* How might additional attention and 
reliance on evidence of student 
learning be encouraged as the central 
feature of quality for both longer and 
shorter-term experiences? 


Ensuring easily accessible and reliable 
information on the quality features of 
shorter-term educational experiences is 
also a challenge for the years ahead. No 
doubt private portals will discover or expand 
this niche.2° Professional associations will 
make sure to inform their membership.” 
Universities, alternative providers, quality 
assurance bodies and accreditors all will 
need to take responsibility and provide their 
stakeholders and end users with relevant, 
complete and up to date information on the 
results of reviews carried out.” 


20 StudyPortals, the Global Study Choice Platform has set up a Short Courses Portal. The new Europass online 
platform will provide access to learning opportunities Europe wide. 


21 Engineers Europe is planning a peer review platform for online engineering courses. 


22 The European Quality Assurance Register for Higher Education (EQAR) is building a Database of External Quality 


Assurance Reports (DEQAR). 
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